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Technical Field of the Invention 



The invention relates generally to image processing and, more specifically, to 
noise reduction for an image sequence. 

Background of the Invention 

It is common for images to appear as a sequence of images, for example, as video 
or motion pictures. The display of the sequence of images in rapid succession may be 
used to give the impression of continuous fluid motion. However, as with the processing 
of any signal, the processing of signals representing image sequences generally 
introduces noise into the image sequences. Such processing may include recording, 
storing, manipulating, transmitting, and displaying of the image sequences. For example, 
noise may result from weak television signals, a poor cable television connection, aging 
and handling of motion picture film, or during playback of a videocassette tape in a 
videocassette recorder. Thus, as more processing is performed on an image sequence, the 
noise introduced into the image sequence accumulates. Consequently, the image 
sequence displayed is not a "perfect copy" of the original image sequence, but is 
progressively degraded as the image sequence is processed. 

Thus, it is desirable to reduce the noise in the image sequence. Moreover, image 
sequences are typically processed at a high rate, for example 24 or 30 frames per second. 
Each frame constitutes a separate image may include thousands or millions of pixels. 
Thus, if processing of each frame or each pixel is computationally complex, the amount 
of processing power needed may easily become overwhelming. Thus, a technique is 
needed which provides high quality image sequences by providing effective noise 
reduction, but which can be implemented using a few simple numerical operations. 



Brief Description of the Drawings 

FIG. 1 is a block diagram illustrating an embodiment of the present invention. 

FIG. 2 is a diagram illustrating a blending factor 202 as a function of difference 
201 in accordance with an embodiment of the present invention. 

FIG. 3 is a block diagram illustrating an apparatus in accordance with an 
embodiment of the present invention. 

FIG. 4 is a flow diagram illustrating a method in accordance with an embodiment 
of the present invention. 

FIG. 5 is a block diagram illustrating a program storage device and a machine in 
accordance with an embodiment of the invention. 
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Detailed Description of a Preferred Embodiment 

A method and apparatus for reducing noise in an image sequence is provided. 
Images in the image sequence are recursively filtered on an area-by-area or, for example, 
5 pixel-by-pixel, basis. A pixel from an image is compared to a similarly located pixel 
from a previous image. A difference between one or more parameters of the pixels is 
determined. One or, preferably, two thresholds are used to classify three types of 
differences. Depending on the classification, the two pixels are blended together 
according to their parameters in varying amounts. Relatively small differences indicate a 
10 large fraction of the previous pixel to be combined with a small fraction of the current 
pixel. This substantially reduces the effects of random noise, which tends to cause 
relatively small, transient variations in the pixels of each image of the image sequence. 
O Relatively large differences indicate the use of 100% of the current pixel without 

combining any of the previous pixel Intermediate differences indicate smoothly 

[y 15 transitioning from combining a large fraction of the previous pixel to combining none of 
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m the previous pixel depending on the differences relative to the thresholds. 
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FIG. 1 is a block diagram illustrating an embodiment of the present invention. A 
previous property 108 of a previous instance 105 of a pixel from previous image 101 is 
20 provided to noise reducer 104. A current property 109 of a current instance 106 of a 
pixel from current image 102 is provided to noise reducer 104. A fixed ratio 1 1 1, a 
higher threshold 1 12, and a lower threshold 1 13 are provided to noise reducer 104. Noise 
reducer 104 determines an absolute difference between the previous property 108 and the 
current property 109. Noise reducer 104 compares the absolute difference to a higher 
25 threshold 112 and a lower threshold 113. Based on the comparisons, noise reducer 104 
determines a new property 1 10 for a new instance 107 of a pixel in a new image 103. 
Depending on the absolute difference, noise reducer 104 may use fixed ratio 1 1 1 in this 
determination. New image 103 may be used to replace current image 102, with new 
image 103 serving as a reduced-noise version of current image 102. Thus, new image 
30 103 may be displayed rather than current image 102. 
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Moreover, to apply this embodiment of the invention to subsequent images of the 
image sequence, what is referred to here as the new image 103 is used as the "previous 
image," while the subsequent image is used as the "current image" to yield a noise- 
reduced version of the subsequent image. Thus, the process uses as one of its inputs a 
5 property of a pixel from an image that has already been processed using the process. 

Therefore, the process serves as a recursive filter, or infinite impulse response (IIR) filter, 
to filter noise from the image sequence. 

FIG. 2 is a diagram illustrating a blending factor 202 as a function of difference 
201 in accordance with an embodiment of the present invention. Line 207 represents 
100% of the value of then new property calculated by the noise reducer. The new 
property is determined as a combination of previous property 208 and current property 
209 based on a blending factor 202 as a function of a difference 201. If the difference 
201 is less than a lower threshold 203, fixed ratio 205 applies. With fixed ratio 205, the 
portion of previous property 208 below fixed ratio 205 is added to the portion of current 
property 209 above fixed ratio 205 to determine the new property. If the difference 201 
is greater than lower threshold 205 but less than higher threshold 204, a variable ratio 206 
is applied. Variable ratio 206 linearly decreases between lower threshold 203 and higher 
threshold 204. At higher threshold 204, variable ratio 206 reaches a zero percent 
blending factor, where none of previous property 208 is used to determine the new 
property. If the difference 201 is above higher threshold 204, the new property is entirely 
dependent on current property 209 and is independent of previous property 208. 

Thus, if the difference between the previous instance of a pixel and the current 
instance of a pixel is small (i.e., less than lower threshold 203), previous property 208 is 
used to determine a large percentage of the new property, while the current property 209 
25 is used to determine only a small percentage of the new property. Thus any slight 

variations in the current property that result from noise have much less influence over the 
new property, thereby reducing noise. If the difference between the previous property 
and the current property is large (i.e., greater than higher threshold 204), current property 
209 is used to determine the new property. Thus, for large changes, which may represent 
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completely different information, for example a scene change of a video image, current 
property 209 is used since previous property 208 may be obsolete and complete 
independent of current property 209. By basing the new property on current property 209 
when large difference occur, residual information that is no longer relevant to the pixel is 
5 eliminated and the new property accurately reflects the new state of the pixel. 

In the region between lower threshold 203 and higher threshold 204, it is more 
difficult to distinguish between the influence of noise on the pixel and the influence of 
other events, such as a scene change or a change in an object represented by the pixel. 
Thus, variable ratio 206 is used to adjust the amount of previous property 208 blended 
10 with current property 209 to determine the new property. The linear transition of variable 
ratio 206 from fixed ratio 205 to a zero blending factor helps smoothly accommodate 
application of the invention from pixels which may have been influenced by noise to 
pixels which have been changed intentionally. 

FIG. 3 is a block diagram illustrating an apparatus in accordance with an 
15 embodiment of the present invention. A previous property 310 is provided to an absolute 
difference calculator 301 and a new property calculator 303. A current property 3 1 1 is 
provided to the absolute difference calculator 301 and the new property calculator 303. 
The absolute difference calculator 301 determines an absolute difference 323 (i.e., the 
absolute value of the difference) of the previous property 310 and the current property 
20 311. The absolute difference calculator 301 provides absolute difference 323 to higher 
threshold comparator 305 and to lower threshold comparator 306. A higher threshold 
312 is provided to higher threshold comparator 305. A lower threshold 313 is provided 
to lower threshold comparator 306. An output 316 of higher threshold comparator 305 is 
provided to an input of NOR gate 307 and blending factor selector 309. The output 316 
25 indicates whether the absolute difference 323 is less than or greater than the higher 
threshold 312. 

Lower threshold comparator 306 compares absolute difference 323 and lower 
threshold 313. Lower threshold comparator 306 provides an output 317 that indicates 

6 




whether absolute difference 323 is less than or greater than lower threshold 313. Output 
317 is provided to fixed ratio block 314. Fixed ratio 315 is provided to fixed ratio block 
314. Output 3 1 8 of fixed ratio block 3 14 is provided to blending factor selector 309. 
Output 317 of lower threshold comparator 306 is also provided to an input of NOR gate 
5 307. An output 319 of NOR gate 307 is applied to blending factor calculator 308. 

Blending factor calculator 308 calculates a blending factor when absolute difference 323 
lies between lower threshold 313 and higher threshold 312. Output 320 of blending 
factor calculator 308 is provided to blending factor selector 309. Blending factor selector 
309 selects the appropriate blending factor and provides an output 321 to new property 
10 calculator 303. New property calculator 303 uses the blending factor, in conjunction with 
the previous property 310 and the current property 31 1, to determine a new property 322. 
New property calculator provides new property 322 to be used by display 304 to display 
a pixel representing the new property 322. The new property 322 may be stored to allow 
its use as previous property 310 in a subsequent iteration for a subsequent pixel. 

FIG. 4 is a flow diagram illustrating a method in accordance with an embodiment 
of the present invention. The method begins in step 401 and continues to step 402. In 
step 402, a current property for a current instance of a pixel is obtained. From step 402, 
the process continues to step 403. In step 403, a difference between a previous property 
for a previous instance of the pixel and the current property is determined. From step 
403, the process continues in step 404. In step 404, a determination is made as to 
whether or not the difference is greater than a higher threshold. If the difference is 
greater than the higher threshold, the process continues in step 405. In step 405, a new 
property is produced using the current property. This occurs independent of the previous 
property. From step 405, the process continues to step 409, where it ends. 

25 If, in step 404, it is determined that the difference is not greater than the threshold, 

the process continues to step 406. In step 406, a determination is made as to whether the 
difference is less than a lower threshold. If the difference is less than a lower threshold, 
the process continues to step 407. In step 407, the new property is determined by 
combining the previous property and the current property in a fixed ratio. The fixed ratio 
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is expressed as a previous-property-to-current-property-ratio. In one embodiment of the 
invention, the fixed ratio is preferably greater than 1:1. The invention may be practiced, 
for example, with the fixed ratio greater than 2:3, between 13:7 and 19:1, between 7:3 
and 9:1, or between 3:1 and 17:3. From step 407 the process continues to step 409, 
5 where it ends. 



If in step 406, the determination is made that the difference is greater than the 
lower threshold, the process continues to step 408. In step 408, the new property is 
determined by combining the previous property and the current property in a variable 
ratio, with the variable ratio depending on the difference. The variable ratio is more 
10 heavily weighted toward the current property than is the fixed ratio. The variable ratio 
exhibits an inverse linear relationship with the absolute difference between the lower 
Q threshold and the higher threshold. From step 408, the process continues to step 409, 

S where it ends. 

W 
m 

yj The properties processed according to the method may be one or more individual 

: m 

j}j 15 pixel parameters, an aggregation of pixel parameters, or a value that characterizes a pixel 
L_ as a whole. Examples of pixel parameters that may be processed include a red parameter, 

SJ a green parameter, a blue parameter, a cyan parameter, a magenta parameter, a yellow 

parameter, a chrominance parameter, a luminance parameter, a transparency parameter, a 

y texture parameter, a grayscale parameter, an opacity parameter, a color parameter, a fog 

□ 

20 parameter, a lighting parameter, a layer parameter, and a depth parameter. Any other 
identifiable properties of a pixel may be processed according to the invention. For 
example, a bit pattern representative of the pixel or some property thereof may be 
processed according to an embodiment of the invention. 



Moreover, the invention may be practiced using only the lower threshold or only 
25 the higher threshold instead of using both thresholds. Alternatively, additional thresholds 
beyond the lower threshold and the higher threshold may be introduced. For example, a 
third threshold may be introduced between the lower threshold and the higher threshold. 
Different variable ratios may be applied on each side of the third threshold. Also, the 
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variable ratio or ratios need not be linear. Rather, a non-linear transition may be provided 
between the lower threshold and the higher threshold. For example, a variable ratio may 
be configured to provide a non-linear curve between the lower threshold and the higher 
threshold. Examples of such non-linear curves include exponential and logrithmic 
5 curves. 



A variety of values may be used for the fixed ratio. These ratio values may also 
be expressed as blending factors, where the blending factor indicates the percentage of 
the new property based on the previous property. For example, if the previous property 
10 has three times the influence of the current property on the new property, the ratio would 
be expressed as 3:1 and the blending factor would be expressed as 75%. Examples of the 
^ fixed ratio include a fixed ratio greater than 1 : 1 (a blending factor greater than 50%), a 

■J\ fixed ratio greater than 2:3 (a blending factor greater than 40%), a fixed ratio between 

p 5 ! 13:7 and 19:1 (a blending factor between 65% and 95%), a fixed ratio between 7:3 and 

jjj 15 9:1 (a blending factor between 70% and 90%), and a fixed ratio between 3:1 and 17:3(a 

; r | 

fn blending factor between 75% and 85%). 

O Likewise, a variable ratio may span various ranges of values. For example, for 

pj example, the variable ratio may be included in a range from 0: 1 (a blending factor of 0%) 

20 to the fixed ratio. The variable ratio may exhibit an inverse linear relationship with the 
O absolute difference. Since the variable ratio is applicable between the lower threshold 

and the higher threshold, the inverse linear relationship is observed between these 
thresholds. According to the inverse linear relationship, the variable ratio decreases 
linearly as the absolute difference increases between the lower threshold and the higher 
25 threshold. 



FIG. 5 is a block diagram illustrating a program storage device and a machine in 
accordance with an embodiment of the invention. Machine 501 is coupled to program 
storage device 502 via coupling 504. Program storage device 502 includes a program of 
30 instructions 503. The program storage device 502 is readable by the machine 501 . The 
program storage device 502 tangibly embodies the program of instructions 503 
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executable by the machine 501 to perform a method of motion estimation for encoding 
sequential frames. The method may includes the steps recited with reference to FIG. 4. 
The machine 501 may be a general purpose computer system or a component thereof. 
Alternatively, the machine 501 may be specifically dedicated to performing the method 
steps described above. 

One advantageous aspect of the invention is that is does not require steps to be 
taken before noise is introduced into the image sequence. Rather, the invention may be 
applied to image sequences into which noise has already been introduced. Thus, the 
invention is broadly applicable to image sequences affected by noise. 

Accordingly, a method and apparatus for reducing noise in an image sequence has 
been described. It should be understood that the implementation of other variations and 
modifications of the invention in its various aspects will be apparent to those of ordinary 
skill in the art, and that the invention is not limited by the specific embodiments 
described. It is therefore contemplated to cover by the present invention, any and all 
modifications, variations, or equivalents that fall within the spirit and scope of the basic 
underlying principles disclosed and claimed herein. 
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